Adaptive web caching: towards a new global caching architecture

An adaptive, highly scalable, and robust web caching system is needed to effectively handle the exponential growth and extreme dynamic environment of the World Wide Web. Our work presented last year sketched out the basic design of such a system [31]. This sequel paper reports our progress over the past year. To assist caches making Web query forwarding decisions, we sketch out the basic design of a URL routing framework. To assist fast searching within each cache group, we let neighbor caches share content information (borrowing work elsewhere on efficient hashing functions). Equipped with the URL routing table and neighbor cache contents, a cache in the revised design can now search the local group, and forward all missing queries quickly and efficiently, thus eliminating both the waiting delay and the overhead associated with multicasted queries. The paper also presents a proposal for incremental deployment that provides a smooth transition from the currently deployed cache infrastructure to the new design.